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The use of on-line instructional delivery techniques such as instructional web sites, on- 
line communication tools, and multimedia productions to deliver classes is a rapidly growing 
practice in education. It has been suggested that most universities offer some courses using on- 
line technology (Beller & Or, 1998). Recent literature related to effectiveness of and 
satisfaction with on-line instruction focused on reasons to implement on-line instruction as a 
replacement for traditional face-to-face instruction and on comparing student achievement and 
attitudes toward this approach to delivery. Various reasons have been noted for delivery of 
instruction on-line. 

Beller and Or (1998), and Johnson, Aragon, Shaik, and Palma-Rivas (2001) discussed the 
possibility that on-line learning increased enrollments due to the convenience encountered by 
students at a distance. Expectations of cost effectiveness for both students and universities 
were discussed by Beller and Or, and by LaRose, Gregg, and Eastin (1998). It has also been 
suggested that on-line learning has the potential to enhance individual student learning 
(Kandies & Stern, 1999; LaRose, Gregg, & Eastin, 1998). Lastly, Sanders and Morrison- 
Shetlar (2001) discussed the importance of on-line learning as a means to assure student 
familiarity with computer technologies. 
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Research thus far has primarily focused on identifying factors that lead to success in on- 
line classes as an alternative to traditional instructional delivery. However, what has not 
occurred to date is sufficient exploration of those factors in the context of on-line instruction 
embedded in an otherwise traditional instructional environment. 

Findings of previous studies have indicated that students participating in classes entirely 
delivered through electronic distance learning formats have no significant differences in 
learning when compared to students taking classes in traditional classroom settings (Johnson, 

Aragon, Shaik, & Palma-Rivas, 2001; LaRose, Gregg, & Eastin, 1998; Swan & Jackman, 

2000). Students participating in studies of an entire class delivered on-line seemed to enjoy the 
convenience of asynchronous class delivery. Further studies indicated that students have been 
mixed in their level of satisfaction with teacher interaction when enrolled in on-line courses 
(Chester & Gwynne, 1998; Johnson, Aragon, Shaik, & Palma-Rivas; LaRose, Gregg, & 

Eastin) and with their ability to create communities of learning with other dispersed students 
through tools used in on-line delivery (Benbunan-Fich & Hiltz, 1999; Johnson, Aragon, Shaik, 

& Palma-Rivas). 

Further, findings on courses delivered electronically indicated that enjoyment and control 
of pace were significant factors (Fitzelle & Trochim, 1996). Christensen, Anakwe, and Kessler 
(2001) found that students whose attitude toward the usefulness of technology was positive 
were significantly more receptive to on-line instruction than those students with a negative 
attitude toward the usefulness of technology. They also found that technology familiarity was 
positively related to receptivity for on-line instruction, and that technology accessibility was 
not significantly related to receptivity for on-line instruction. 

Studies examining student attitudes and learning related to web-enhanced instruction 
found that students participating in a web-enhanced class had mixed levels of satisfaction with 
teacher interaction and with reactions to their ability to create learning communities through 
on-line tools (Sanders & Morrison-Shetlar, 2001; Benbunan-Fich & Hiltz, 1999). Sanders and 
Morrison-Shetlar found females to be significantly more positive toward web-enhanced 
instruction than males. In neither study did age relate significantly to attitude toward web- 
enhanced instruction. Benbunan-Fich and Hiltz found that asynchronous learning networks 
enhanced the product quality in collaborative learning projects. 

Bandura (1994) described self-efficacy as beliefs about personal capabilities to perform 
in ways to exercise influence over events that affect our lives with respect to cognitive, 
motivational, affective, and selection processes. Self-efficacy in this study was operationally 
defined as representing student sense of being personally in control of factors such as time, 
pace of instruction, and the technology (software and computer hardware). Self-efficacy has 
been suggested as an important factor in students' efforts to use and succeed with electronic 
technologies (Eastin & LaRose, 2000; Kandies & Stern, 1999). 

Purpose of Study 

Knowledge of how student a priori experiences with computer technologies and 
applications correlate to student perceptions of and success with embedded on-line learning 
experiences could enhance decisions about the design and delivery of on-line components 
embedded in traditional classroom instruction. For the purpose of this study, student success 
was operationally defined as achieving learning objectives as indicated by test and project 
scores. This knowledge could also begin to provide the basis for development of a useful pre- 
assessment instrumentation to assist instructors in identifying which students are most likely 
to be successful in achieving learning objectives with embedded on-line instruction. More 
specifically, it could be useful to better understand how students react when selected portions 
of course material are delivered on-line to support instruction in an otherwise traditional 
classroom. With that in mind, the following research questions were asked. 
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1. What student characteristics and a priori experiences correlate with student perceptions 
of satisfaction with on-line learning experiences embedded in a traditional classroom 
environment? 

2. What student characteristics and a priori experiences correlate with student success in 
achieving learning objectives when participating in on-line learning experiences 
embedded in a traditional classroom environment? 

Methodology 

At the beginning of each semester, prior to introduction of the embedded on-line 
instructional segment, a pretest instrument was administered to university technical students 
who volunteered to participate in the on-line instructional activity. The opportunity to 
participate in this study was offered to students during two consecutive semester sessions of an 
introductory industrial computer systems class. 

The pretest instrument consisted of four parts. Part one consisted of the 10-item scale 
developed by Compeau and Higgins (1995) to measure computer self-efficacy. Computer self- 
efficacy has been described as the individual's judgment of or belief in his/her own capacity to 
use a computer and is believed to influence personal decisions about what behaviors to 
attempt, as well as persistence (Compeau & Higgins). This scale had been validated in a 
regression study involving more than 1000 participants. Individuals who were higher in 
computer self-efficacy were found to experience less anxiety about attempting a new computer 
application, and to derive more enjoyment from use of the computer (Compeau & Higgins). 

Part two measured the construct described as cognitive playfulness utilizing the seven- 
item Microcomputer Playfulness Scale developed and validated by Webster and Martocchlo 
(1992), involving more than 400 participants over multiple studies. The scale resulting from 
this research measures the trait of cognitive playfulness in the context of microcomputer use. 

The computer playfulness scale (CPS), defined as representing "the degree of cognitive 
spontaneity in microcomputer interactions" (Webster & Martocchlo, p. 201), focused on seven 
characteristics of cognitive spontaneity: sense of feeling spontaneous, imaginative, flexible, 
creative, playful, original, and inventive when using a personal computer. This research 
suggested that individuals who were higher in the trait microcomputer playfulness would be 
more spontaneous, imaginative, and inventive in their microcomputer interactions. It was 
further suggested that the trait described as high cognitive computer playfulness can be 
motivating and relate positively to learning in the context of computers; significant positive 
correlations were found between learning outcomes and computer playfulness (Webster & 
Martocchlo). 

Part three consisted of two scales of six items each designed to assess the constructs of 
perceived usefulness and perceived ease of use. These scales were developed, refined, and 
validated by Davis (1989). Cronbach alpha reliability for the usefulness scale was reported at r 
= .98 and for the ease of use scale at r = .94. Perceived usefulness is defined as the degree to 
which individuals believe a technology will help them perform better. Perceived ease of use 
was defined as the degree of difficulty an individual believes would be required in order to use 
a technology. Both scales exhibited high convergent, discriminant, and factorial validity; thus, 
construct validity was demonstrated. Both ease of use and usefulness scale results were found 
to have strong correlations with actual use of technologies, but the usefulness scale was found 
to be a better predictor of future use (Davis). 

Cronbach alpha scores were calculated for the scales comprising parts one, two, and three 
of the pretest instrument. Cronbach alpha for part one of the instrument, computer self- 
efficacy, was r = .907; and for part two, computer playfulness, r = .914. Cronbach alpha for 
the first half of part three, computer usefulness, was r = .892, and for the second half of part 
three, computer ease of use, r = .948. 
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Part four was created by the researchers and collected information about previous student 
familiarity with computers and specific software applications. Some of these software 
applications included web browsers, word processors, e-mail, and presentation software. 

Following the completion of the embedded on-line instructional segment, a second 
questionnaire, previously designed by Ryan, Hodson Carlton, and Ali (1998) to specifically 
assess student perceptions about on-line instruction, was administered to the students who 
participated in the three-week on-line learning experience. Ryan, Hodson Carlton, and Ali 
determined the reliability of this instrument using Cronbach's alpha to be r = .82 for the 
WWW module scale and a test-retest procedure. The Likert-type items on the second 
questionnaire ranked from (1) strongly agree to (5) strongly disagree. This questionnaire 
solicited student perceptions of the on-line learning environment. Specifically, the 
questionnaire included items about student perceptions on how well and the extent to which: 

• Content was covered, 

• Interaction was evident (among students and between students and faculty), 

• Student participation was facilitated, 

• Communication skills were required, 

• Critical thinking was required, 

• Time was appropriate for assignments, 

• Faculty preparation and expertise was important, and 

• Technical skills were required 

In addition, learning outcomes were assessed via both on-line tests and performance- 
based exercises throughout each three-week on-line block. 

Information received from both the pre- and post- assessment questionnaires was 
matched for individual students, along with grades on tests and exercises. Only students who 
provided data in both the pre- and post-assessments were included in the final analysis. The 
pre-assessment was administered at the beginning of the semester. The three-week on-line 
instructional block took place in the middle of the semester. Correlations (Pearson's R) were 
used to identify relationships within the data. An Alpha level of p > .05 was applied. 

Population 

For two semesters (fall 2001 and spring 2002), students enrolled in an introductory 
industrial computer systems class were eligible to participate in the project. For each semester, 
student volunteers agreed to participate in one on-line instructional module to support the 
study. During the remainder of the semester, students were taught using traditional 
instructional methods. All nonparticipating students were taught entirely using traditional 
classroom-based methods. The data received from the two on-line cohorts were combined, 
allowing for a sample of 57 students. 

Findings and Discussion 

This section presents and discusses correlations between pre-assessment survey 
items/constructs and post-treatment learning outcomes, as well as students' perceptions of their 
on-line learning experience as assessed through the post-instruction assessment instrument. 

These correlations are reflected in Table 1. The pre-assessment instrument asked for 
indications of a priori computer technology experiences as well as responses to items designed 
to assess the constructs of computer self-efficacy, computer playfulness, perceived computer 
usefulness, and perceived ease of use. 

Contrary to suggestions in the literature, no statistically significant correlations were 
found. The most surprising finding here was that self-efficacy did not emerge as a significant 
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characteristic. Since computer self-efficacy was believed to relate to the individual's 
belief that he/she can master a challenge, this result may indicate that students did not view 
learning in the context of interactive web-based instruction embedded within an otherwise 
traditional course as more challenging or difficult that the traditional instructional method. 

In previous research, ease of use scale results were found to have statistically significant 
correlations with actual use of technologies (Davis, 1989). In this study, the ease of use 
construct was found to have significant correlations with three post-treatment variables: 
student perceptions that during the web-based instructional unit (a) content was adequately 
covered (r = .369), (b) interaction was evident (r = .313), and (c) communication skills were 
required (r = .291). This may suggest that students found the use of synchronous chats, 
asynchronous bulletin boards, e-mail, and voice-over PowerPoint presentations adequate 
communication tools in this teaching/learning environment. 

The relationship between a priori familiarity with computers and specific software 
applications, and student perceptions of satisfaction with on-line learning experiences 
embedded in a traditional classroom environment, was positively supported in one variation. It 
had been speculated that prior experience and perceived proficiency with such computer 
applications as Microsoft Windows, Internet browsers, electronic 


Table 1 

Correlations between Pre and Post-online Instruction Items 



Power 

Point 

e-mail 

Cognitive 

playfulness 

Computer 

ease 

Computer 

usefulness 

Quiz 

.275* 

.143 

.100 

.097 

.272* 

Time 

.026 

.395* 

.032 

.214 

.168 

Interaction 

.208 

.071 

.284* 

.313* 

.374* 

Content 

.067 

.034 

.244 

.369* 

.341* 

Communication 

.158 

.026 

.079 

.291* 

.291* 

Critical thinking 

.137 

.237 

.153 

.150 

.358* 

* significant at the .05 

evel 


mail, and PowerPoint would correlate with the student's perceptions about the web-based 
instructional unit that incorporated elements similar to those in each of these applications. One 
positive statistically significant correlation of r = .395 was observed between student 
experience with use of e-mail and post-treatment perceptions that the time available during the 
web-based instructional unit was appropriate for the assignments. This would suggest that 
proficiency with e-mail may be an important prerequisite skill for students participating in 
asynchronous web-based instructional units. 

The relationship between cognitive playfulness and success in achieving learning 
objectives with on-line learning experiences embedded in a traditional classroom environment 
was supported. Based on previous findings that high cognitive computer playfulness was 
potentially motivating and related positively to learning (Webster & Martocchlo, 1992), it was 
expected that there would be positive statistically significant correlations between this 
construct and learning outcomes. The playfulness construct was found to significantly 
correlate with only one post-treatment measure. The correlation between computer playfulness 
and student perceptions that interaction was evident (among students and between students 
and faculty) during the web-based instructional segment was r = .284. This suggests that 
students who enjoy using computers may also value interpersonal communications facilitated 
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by the technology. 

The relationship between perceived usefulness of personal computers and student success 
in achieving learning objectives with on-line learning experiences embedded in a traditional 
classroom environment was supported. Previous research suggested that the computer 
usefulness construct was a useful predictor of future use of computer technologies. The 
computer usefulness construct was significantly correlated with five post- treatment variables: 
learning outcomes as measured by cognitive quiz and unit test scores (r = .272), as well as 
student perceptions that while completing the web-based instructional unit (a) content was 
adequately covered (r = .341), (b) interaction was evident (r = .374), (c) critical thinking was 
required to successfully complete the assignments (r = .358), and (d) communication skills 
were required (r = .291). 

The relationship between a priori familiarity with computers and specific software 
applications, and student success in achieving learning objectives with on-line learning 
experiences embedded in a traditional classroom environment, was supported in one variation. 

A statistically significant correlation was found relating a priori student experience in use of 
PowerPoint with cognitive learning outcomes as demonstrated through scores of unit tests and 
quizzes. A positive statistically significant correlation of r = .275 was found. Since the 
delivery format for the quizzes on line was via PowerPoint presentation slides saved to the 
instructional web site, it appears that students believed that proficiency with PowerPoint may 
have also been an important prerequisite skill for students participating in these web-based 
instructional units. 


Conclusions 

Typical reasons noted for delivery of whole course instruction on-line are convenience 
for students resulting in increased enrollments, expectations of cost effectiveness, enhancing 
individual student learning, and assuring student familiarity with computer technologies. 
Increasing enrollments and cost reductions do not seem to be relevant to decisions on how to, 
or even whether to, embed on-line components within a traditional classroom environment. 
However, the potential for enhancing learning for some or even all students and assuring 
student familiarity with computer technologies may be more compelling. Further study seems 
warranted to determine the degree to which embedded on-line instruction may enhance student 
motivation and learning for select topics and/or students who exhibit specific characteristics. 
The question also arises whether there are effective pre-assessment themes and variables that 
could indicate which, if any, students will find that on-line instruction enhances learning. 

The primary implications of this study are that it is essential for on-line component 
instructors to consider student characteristics and preferences before delivering on-line 
instructional segments. It must be remembered that both learning and using computers to aid 
learning and productivity can and should be enjoyable. Nevertheless, students learning new 
skills with computer technology and using a new software package to complete new tasks face 
numerous challenges. When developing on-line components embedded within traditional 
instructional environments, instructors need to reinforce student perceptions that the 
technology is useful and make learning activities and use of computers to support learning 
enjoyable. 

The findings of this study suggest several possible areas for further study. 

1. Some of the variables selected for inclusion in this study, specifically, computer self- 
efficacy and computer playfulness, do not appear to be important factors related to 
students' perceptions about imbedded web-based instruction. 

2. The constructs of computer ease of use and computer usefulness may warrant further 
study to see if they have any predictive value in anticipating student learning success 
and attitudes toward web-based imbedded instructional units. 
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3. Further exploration is needed if constructs which have adequate predictive value when 
anticipating student learning success and attitudes toward web-based imbedded 
instructional units are eventually to prove useful. 

Students who believe that computers are useful, easy to learn, and flexible seem more 
likely to be computer literate and have sufficient knowledge to focus on the instructional 
content rather than becoming preoccupied with learning to use the on-line technology. Finally, 
on-line instruction may provide opportunities to teach with a variety of different learning style 
preferences and to shape instructional experiences that are beneficial to learners in a world of 
increasing diversity in communication and learning styles. 
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